
On the day of testing we handed script to our testers and asked them to follow those scripts. After users completed working with scripts there were given some time to use the system as they wished.
While testing, we would make sure there were multiple users using the system at a time to check how the system would cope. During testing we were making notes on what tester were doing and if they had any difficulties using the system. This was to allow us to assess the usability of the user interface and its easiness of use.
We would also make notes of any tasks users could not perform or functions that were not working properly also noting what was happening exactly. In addition to that, we would take any other feedback from the user and any suggestion they had.

While testing the system there was a number of issues that our evaluators pointed out, some of which we were already aware of. We also received a lot of useful feedback about the system and its usability from our evaluators.

While testing the system it appeared that the registration process fails. User is able to display the registration window and submit his/her data but no data is actually stored after the submission and so the user cannot log in after the registration.

Similarly, within the main window, if the user decided to change his personal information such as name, or display name no data is stored on the server side. 

Also, when user changes the status or an avatar that change is only shown to the user himself and not to all of the contacts on his contact list.

In addition, when uploading an avatar it is possible for the user to upload any type file as no restriction on the file types has been implemented. However, when the user attempts to upload a file of a type other than an image file the system will thrown an exception. 

Furthermore, it is not possible to search for and add users to the contact list as well as it is not possible to remove users from the contact list as those features have not yet been implemented fully. For this reason during testing evaluators had to use a default contact list that was hard-coded in the system.

Similarly, it is also not possible to create channels and invite users to them. Again, those features were not implemented as other functionality that had to be developed prior to this was not working correctly. In effect, channel communication has not been tested.

As for sending messages the system is usable in that it sends messages to correct people/chat windows and it is possible to have multiple conversation at the same time.
However, there are still a number of problems that arise during conversations.

The most obvious of issue is that messages between the users are displayed in the wrong order, i.e. the most recent message is shown at the top of the window instead of at the bottom. This is quite inconvenient not only because it does not correspond to displaying messages by other IM system but also because when the conversation gets long the user has to scroll up the window to see the most recent message everytime it is received.

Also, some redundant characters are displayed with every message received such as “:” or newlines. Unfortunately we failed to establish what is causing this.

If the user attempts to send an emoticon within his message that emoticon will be displayed to the sender but not to the receiver. This is because along with the message a path to that specific emoticon is send as well. However, the path is only correct for the sender hence the receiver cannot find such path resulting in no image being displayed.

Moreover, it is possible to type in HTML code into the message window. This is necessary to allow sending emoticons or formatted text. However, should sender type in an HTML code for changing the colour of the chat window, displaying buttons or search boxes etc. all of those will be accepted by the system and will be displayed in the chat window accordingly. The more serious problem arises when sender or a receiver attempts to, for example, click on such a button. This causes the system to freeze and no messages can be send or received.

One of the more serious issues is caused by changing the colour of the main window. When users attempt to change that setting all of his current conversations are lost and it is no longer possible to send or receive any messages.

Apart from finding out what features of the system do not work correctly, our evaluators also provided us with a small number of suggestion on how the system could be improved in the feature.

During registration when user presses the submit button the window closes completely leaving the user unaware of the state of the system. Instead, after the registration data is submitted the user should be send back to the login window and perhaps a message confirming the successful (or not) registration could be displayed.

Similarly, if users provides an incorrect data at login, such as username and passwords that do not match, the login window refreshes allowing user to try again. However, again no message is being displayed informing the user of what happened.

Also within the main window there were a number of issues. 

When using the system for the first time it was unclear to users how to start sending messages, i.e., the solution to double click on a contacts name was not as intuitive as we first thought.
Similarly, it was necessary to explain to users how to receive messages as they did not expect the sender’s name to be highlighted when a message has been received, neither did they know to double click on the highlighted contact in order to read the message. 
Also, evaluators complained about the highlight colour being black, as they cannot see who currently send them a message. Unfortunately, the colour used for highlights is system-dependent and all of our attempts to overwrite it failed.

In addition, evaluators pointed out that they would like to be able to see all people they are currently talking to (if having multiple conversations). This would ideally be achieved by using more than one tab. Users also expressed the need for the ability to close any tabs in order to finish a conversation.

Furthermore, one of the evaluators pointed out that the window displayed to change the text format is slightly overwhelming. He suggested that the window could be organised better by either using tabs to separate colour of the text from its style or by only displaying the basic methods to change the text with an option to apply more advanced changes if a user wish to do so. 

Also, one of the tester suggested that it would be useful to have a way of entering a newline into the message. At the moment the “enter” button sends the message to the chat window and there is no other way of producing a newline (other than submitting an HTML code for it). The user suggested that perhaps having a separate button to send a message could help with the issue. 

The final suggestion made was about the size of the main window. It was pointed out that the initial window which only displays the contact list could have been smaller at first and expanding as necessary when a message is to be send or is received. 

Also the window is a bit sparse when expanded which does not look very aesthetic.  So perhaps all of the components should be re-sized to make the better use of the space.

In general, the evaluators stated that the overall design of the system is simple enough and is relatively intuitive to use.

